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ting  managers  and  accountants  is  design  of  computer  simulation  experiments  in 
settings  where  the  assumptions  usually  made  by  statisticians  are  violated.  One 
approach  to  this  problem  is  to  run  the  simulation  "until  the  budgeted  money  runs 
out  and  hope  for  the  best  analysis  of  the  simulated  results.  The  present  paper 
shows  how,  instead,  to  rationally  design  and  analyze  a  computer  simulation  under 
minimal  assumptions,  using  the  Bishop-Dudewicz  HANOVA  (Heteroscedastic-ANOVA) 
Procedure.  HANOVA  also  allows  the  experimenter  to  control  Type  II  error  in 
addition  to  the  usually-controlled  Type  I  error.  The  new  procedure  is  applied. 
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INTRODUCTION 


Frequently,  after  a  computer  simulation  experiment  with  models  of 
accounting  and  business  systems,  one  or  more  independent  F-tests  are  per¬ 
formed  in  order  to  establish  the  presence  of  model  effects. 

The  traditional  F-tests  assume  that  all  the  population  variables 

under  consideration  have  unknown  means  with  equal  unknown  variances.  The 

experimenter  can  control  Type  I  error  but  not  Type  II  error  under  the 

traditional  F-test  simulation  experiment-^  Previous  accounting  or 

business  simulation  studies  such  as  Demski's  [1967]  [1970]  implementation 

effects,  Sunders' s  [1974]  evaluating  capital  budgeting  models,  Onsi's 

[1975]  simulation  of  organizational  slack,  Fellingham,  Mock,  Vasarhelyi's 

[1976]  information  choice,  and  Magee's  [1976]  analysis  of  alternative 

cost  variance  models  have  encountered  two  methodological  problems: 

(1)  assumption  of  normally  distributed  population  variables  with  unknown 

means  but  equal  unknown  variances  when  most  business  and  accounting 

2 

variables  do  not  come  from  equal  variance  populations;  (2)  inability  to 
control  Type  II  error. 

Lin  [1978]  has  solved  the  first  problem  in  his  simulation  of  multiple 
objective  budgeting  models  but  still  failed  to  control  Type  II  error. 
Unfortunately,  this  means  that  a  failure  to  assert  a  model  effect  in  a 
simulation  of  accounting  or  business  systems  may  be  due  not  to  the 
nonexistence  of  such  effects,  but  rather  to  a  Type  II  error. 

The  objective  of  this  paper  is  to  show  that  the  problem  of 
controlling  Type  II  error  in  accounting  simulation  experiements  can  be 
solved  by  using  a  new  F-test,  which  also  allows  for  unequal  unknown 
population  variances.  The  traditional  analysis  of  variance  (ANOVA)  is 


based  on  the  assumptions  of  normality,  independence  of  the  statistical 
errors,  and  equality  of  the  variances  of  the  errors.  Studies  of  the 
robustness  of  the  F-test  have  shown  that  the  violation  of  normality  has 
little  effect  on  inferences  about  the  means.  However,  the  violation  of 
independence  or  equality  of  variances  can  have  a  serious  effect  on 
inferences  about  the  means,  especially  if  the  cell  sample  sizes  are 
unequal  (see,  for  example,  Scheffe  [1959]  or  Bishop  [1976]).  In  practice, 
the  assumption  of  equality  of  error  variances  seems  to  be  often 
unjustified;  in  fact,  even  when  the  error  variances  are  equal,  the  power 
of  the  F-test  depends  upon  the  unknown  common  variance,  which  renders  it 
difficult  to  plan  an  experiment  rationally.  Recently  Bishop  and  Dudewicz 
[1978]  [l98l]  developed  new  ANOVA  procedures  in  the  contexts  of  the  one¬ 
way'  layout  and  higher-way  layouts.  Their  procedures  allow  unequal  and 
unknown  population  variances  and  give  tests  with  level  and  power 
completely  independent  of  the  unknown  variances. 

The  paper  begins  with  the  background  and  purpose  of  the  study.  The 

second  section  briefly  discusses  a  two-way  layout  heteroscedastic-ANOVA 

(HANOVA)  procedure.  It  is  followed  by  new  study  of  Lin's  [1978]  multiple 

3 

objective  firm  simulation  for  the  two-way  layout  ANOVA  procedure. 
Finally,  the  conclusion  and  possible  business  applications  are  presented. 
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THE  HANOVA  PROCEDURE 


Bishop  and  Dudewicz  [1978]  [1981]  have  developed  heteroscedastic 
ANOVA  (HANOVA)  procedures  in  an  r-way  layout.  For  the  purpose  of  this 
paper,  only  the  two-way  layout  will  be  described. 

The  two-way  layout  under  consideration  is  what  is  usually  studied  in 
ANOVA  and  is  defined  by 


X.  .  =  u  +  a.  +  B.  +  ap.  .  +  e.  .. 

ljk  i  ijk 


(1) 


(i  =  1,  ...,  I;  j  =  1,  ....  J;  k=  1,  2,  ...,  N), 


where  {X^^}  are  the  observed  responses;  it  is  assumed  that  the  {X^^l  are 

independent  and  normally  distributed  with  unknown  mean  E(X^^)  =  p  and  un- 

2 

known  variance  Var(X.  -  a.  .  ;  [e.  ..  }  are  assumed  to  be  independent  ran- 

ljk  ij  ij  k  r 

2 

dom  variables  with  normal  distributions  with  mean  0  and  variance  O. .  .  denoted 

xj 

2  2 

e  ..  ~  N(0,  o  .  ),  0  <  o. .  <  p  is  the  overall  mean;  a.  and  B.  are 

ljk  ij  ij  ’  i 

the  main  effects  of  factors  i  and  j;  aB. .  is  the  interaction  between  i 

ij 


and  j ;  and  that 

J  I 

1 

i=l  '  ]=1  J  i=l 

The  hypotheses  of  interest  are 


1 

1  a 


1  p.  =  1  ap  .  .  =  I  ap. .  =  0. 
J=1  J  i=l  1J  j=l  1J 


H.:  a.  =  0  for  all  i 
0  i 

H,  :  P .  =0  for  all  j 
1  J 

H- :  a  p. .  =0  for  all  i  and  j 


2  •  ij 

In  this  two-way  layout  there  are  I  x  J  possible  treatment 
combinations.  Cell  (i,  j)  refers  to  the  combination  of  level  i  of  the 
first  factor  and  level  j  of  the  second  factor.  One  seeks  tests  of  the 
above  three  hypotheses  based  on  test  statistics  whose  distributions  are 
independent  of  the  unknown  variances. 


(2) 

(3) 

(4) 
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Bishop  and  Dudewicz  [1978]  [1981]  developed  the  following  HANOVA 
procedure  to  test  the  above  hypotheses  based  on  test  statistics  whose 
distributions  are  independent  of  the  unknown  variances  and  both  the  level 
(i.e.,  probability  of  Type  I  error)  and  power  (i.e.,  probability  of 
Type  II  error)  of  the  test  are  controllable: 

1.  Choose  the  Type  I  error  level.  Type  II  error  level,  and  the  tol¬ 
erance  of  the  difference  between  populations. 

2.  Compute  the  design  constant  z  >  0  given  the  parameters  specified  in 
Step  1,  and  then  in  each  cell  (i,j)  take  an  initial  sample  ^ , 


X..„,  X. .  X..  ,  of  size  n  from  each  of  k  populations. 
ij2’  i j  3  ijn^  o 


Compute  sample  variance  the  usual  unbiased  estimate  of  unknown 

2 

population  variances  cr  ^  ,  and  define  the  final  sample  size: 

2* 


N .  .  =  max{ 
ij 


1} 


where  [X]  denotes  the  largest  integer  which  is  smaller  than  X. 


5. 


Take  (N^  -  nQ)  additional  observations  from  cell  (i,j)  populations 


and  calculate  the  weighting  coefficients  a_j, 


aijl  '  ' 
aijno+i 


ijn 


1  -  (N. .  -  n  )  b. . 
ij  o  IJ 

n 


a  .  such  that" 

ijN.  . 
ij 


1 


=  a  .  .  =  b  .  . 

ijN. .  ij 
ij 


n  (N.  .  z 
o  1J 


S.  ,2) 

LI 


and  b ; j  “  V —  ^  1  ^  ? 

J  ^  (Nij  •  no}  Sij 


Compute  cell  sample  generalized  means  X^ 


N.  . 
ij 

1  a.  X .  .. 
ljk  ijk 


where  X.  ,,  ...,  X  , . ,  is  the  final  set  of  observations  for  cell 

ijl’  ijN^ 

(i,i).  Then  compute  group  means  for  treatments  i  and  j: 

x.  =4  i  x. 

J  J=1  lj- 
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X  .  =  y  *  x .  . 

•J.  I  i=1  1J. 


and  grand  mean: 

=  1IJ- 

X.  .  .  =  ~  1  IX 

I J  .  .  ij  . 

i=l  j  =  1  J 


6.  Test  hypotheses  (2), (3),  (4)  based  on  the  following  quadratic  forms 
of  independent,  identically  distributed  Student's  t  variates: 


I  (X  -  X.  .  .) 

j  £  - 


(5) 


i  =  l 


J  (X  .  -  X...)“ 


F> =  1 A 


iA 


(6) 


F  =  I  I 
i=l  j=l 


I  J  (X. .  -  X.  -  X  .  +  X...) 

iJ.  -J- _ _ 


(7) 


_  Of 

The  test  of  proceeds  by  rejecting  if  and  only  if  -  ^"o  n 

..j  j  ,n0 

where  Fq0  is  the  upper  orth  percent  point  of  the  null  distribution  of 
'  o 

Fq.  Tests  for  the  other  hypotheses  are  performed  in  a  similar  manner, 
namely  reject  (or  H^)  if  and  only  if  the  corresponding  statistic  Fj 


th 


~  a 


(or  F^)  is  greater  than  the  upper  a1""  percent  point  Fj“  (or  F^0^  )  of 

’  o  ’  o 

its  respective  null  distribution.^ 

In  general,  this  new  procedure  uses  z  to  replace  traditional  F-test. 's 

2 

a  /N.  The  power  of  the  test  is  one  of  the  inputs  to  determine  the  value 
of  z.  Therefore,  the  Type  II  error  can  be  controlled.  The  new  procedure 
also  uses  generalized  cell  means  X^  to  replace  the  traditional  F-test 
cell  sample  means.  Bishop  and  Dudewicz  [1981]  have  proved  that  } 
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are  independent  random  variables  and  F^,  ,  F^  are  independent  of  . 

Therefore,  the  assumption  of  equal  variances  can  be  eliminated  by  using 
the  new  procedure.  Bishop  and  Dudewiez  [1981]  also  showed  that  the  power 
of  the  test  using  ?  is  much  better  than  the  power  of  the  traditional 
F-test  when  the  variances  are  unequal.  Therefore,  the  new  procedure  is 
more  efficient  than  the  traditional  procedure. 
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MULTIPLE  OBJECTIVE  BUDGETING  SIMULATION:  A  REFINEMENT 


Most  of  the  business  decisions  faced  by  managers  and  accountants  are 
multiple  criteria  or  objective  decision  problems.  Lin  [1978]  conducted  a 
simulation  study  of  a  hypothetical  firm  with  multiple  objectives  of  profit 
and  sales.  He  used  the  traditional  F-tests  to  examine  the  effects  of 
alternative  planning  models  and  accounting  variance  analysis  techniques 
on  the  firm's  profit  and  sales  performance.  The  simulation  results  showed 
that  profit  and  sales  under  multiple  objective  linear  programming  are 
higher  than  those  under  goal  programming  planning  model.  When  comparing 
ex  post  accounting  variance  analysis  with  traditional  variance  analysis, 
the  former  resulted  in  higher  sales,  but  there  was  no  significant  dif¬ 
ference  in  profits.  The  major  reason  was  that  the  observed  sample  means 
were  very  close;  hence,  the  power  of  the  test  was  very  low.  Since  the 
variances  were  unknown,  the  traditional  F-test  was  unable  to  set  its 
sample  size  to  control  the  power.  In  this  paper  we  conduct  a  new  HANOVA 
test.  This  section  will  describe  the  new  HANOVA  test  under  the  two-way 
layout . ^ 

The  main  objective  of  the  simulation  experiment  is  to  study  the 
effects  of  combinations  of  production  planning  models  and  accounting 
variance  analysis  techniques  on  profit  and  sales  performances.  This  study 
uses  a  2  x  2  factorial  design  as  shown  in  Figure  1. 


Planning 
Factor 

2 

Figure  1,  2  Factorial  Experiment 


Variance  Analysis  Factor 


Mode  1 

Mode  2 

Mode  3 

Mode  4 
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The  four  treatment  combinations  are: 


Mode  1  Using  goal  programming  planning  model  and  traditional 

variance  analysis  technique  (GP-TRAD) 

Mode  2  Using  goal  programming  planning  model  and  ex  post 

variance  analysis  technique  (GP-EXPO) 

Mode  3  Using  multiple  objective  linear  programming  planning 

model  and  traditional  variance  analysis  technique  (MOLP- 
TRAD) 

Mode  4  Using  multiple  objective  linear  programming  planning 

model  and  ex  post  variance  analysis  technique  (MOLP-EXPO) 

The  overall  model  is  a  production  planning  decision.  The  planning 
period  is  assumed  to  be  a  month.  The  total  run  length  for  the  short-run 
production  decisions  is  assumed  to  be  two  years  or  twenty-four  periods. 
Sample  size  is  determined  by  replicating  runs  of  this  total  run  length 
using  different  sets  of  pseudo-random  numbers. 

Following  Bishop  and  Dudewicz's  [l98l]  recommended  approximation 
procedure  for  the  two-way  layout,  the  limiting  distribution  of  F^  is  non¬ 
central  chi-square  with  one  degree  of  freedom  and  noncentrality  parameter 


2  '  i^l  “i  2 

Aq  =  - — - - ,  denoted  by  X^(^q)-  Similarly,  it  can  be  shown  that  the 

limiting  distribution  of  is  noncentral  chi-square  with  one  degree  of 


freedom  and  noncentrality  parameter  A^  = 


2  ’  .1,  P 


,  while  the  limiting 


distribution  of  is  noncentral  chi-square  with  one  degree  of  freedom 


2  2 


and  noncentrality  parameter  A^ 
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For  example,  in  this  simulation  the  level  of  the  test,  i.e., 
probability  of  Type  I  error,  was  assumed  to  be  0.05,  with  power  at  least 
0.95,  i.e.,  Type  II  error  probability  =  0.05,  and  the  tolerance  of 

difference  between  modes  for  profit  and  sales  were  assumed  to  be  $30  and 

g 

$100  respectively.  From  Haynam,  Govindara julu  and  Leone's  [1970]  tables 

of  the  cumulative  noncentral  chi-square  distribution,  the  noncentrality 

parameters  are  12.995  for  both  profit  and  sales.  Then  the  standard  errors 

of  e.  timate  z  for  profit  and  sales  are  69.2574  and  769.5267  respectively. 

Taking  initial  sample  size  of  =  30,  the  sample  means  and  variances 
9 

are  shown  in  Table  1. 


Insert  Table  1  Here 


Table  2 


The  final  sample  sizes  and  ether  simulation  statistics  are  shown  in 
10 


Insert  Table  2  Here 


From  the  central  chi-square  table,  the  critical  value  for  Xj  is 

3.84.  From  the  simulation  outputs,  the  actual  profit  under  goal 
programming  is  significantly  different  from  the  profit  under  multiple 
objective  linear  programming  planning  models,  since  F^  =  3.99.  Lin's 
[1978]  study  also  showed  a  significant  difference  of  profit  between  two 
models.  The  actual  sales  under  goal  programming  is  also  significantly 
different  from  the  sales  under  multiple  objective  linear  programming 
planning  models  with  =  75.40.  This  result  is  consistent  with  Lin's 
study.  When  comparing  ex  post  variance  analysis  with  traditional  variance 
analysis,  Lin  [1978]  found  that  the  former  results  in  statistically  higher 
means  of  actual  sales  per  period,  but  that  there  is  no  significant 
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difference  on  the  means  of  actual  profit  per  period.  One  major  reason  for 
nonsignificance  in  profit  was  that  the  power  of  the  test  in  his  study  was 
low.  In  this  study,  the  power  of  test  is  95:% ,  the  simulated  results  (F^  = 
4.85  for  profit  and  5.54  for  sales)  show  that  there  are  statistically 
significant  differences  on  both  profit  and  sales  performance  between 
traditional  and  ex  post  accounting  variance  analysis  techniques. 

In  general,  Bishop  and  Dudewicz's  [1978]  [ 198ll  HANOVA  procedures 
eliminate  the  necessity  of  the  assumption  of  equal  variances.  In 
addition,  the  experimenter  receives  the  added  benefits  of  a  controllable 
power  function  and  allocation  of  additional  sample  size  where  it  is 
important.  The  new  simulation  results  show  that  multiple  objective  linear 
programming  results  in  higher  profit  and  sales  than  those  under  goal 
programming  planning  models.  This  is  consistent  with  the  theory  that  the 
former  is  an  optimizing  model  while  the  latter  is  a  satisficing  model. 
When  comparing  ex  post  variance  analysis  with  traditional  accounting 
variance  analysis,  the  former  results  in  higher  amounts  of  both  profit  and 
sales.  This  result  is  consistent  with  the  theory  that  the  ex  post 
variance  analysis  uses  more  relevant  feedback  information  if  the 
information  is  costless. 
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CONCLUSION 


The  assumption  of  equal  variances  in  ANOVA  is  often  in  doubt,  espe¬ 
cially  when  most  business  and  accounting  variables  do  not  come  from  equal 
variance  populations.  This  paper  introduced  a  new  two-stage  procedure 
which  eliminates  the  necessity  of  such  an  assumption.  In  addition,  the 
experimenter  or  manager  receives  the  added  benefits  of  a  controllable 
power  function  and  allocation  of  the  additional  sample  size  "where  it 
counts"  (i.e.,  where  variability  is  high  and  may  obscure  means).  An 
extension  of  Lin’s  [1978]  multiple  objective  budgeting  simulation  has 
been  illustrated  for  two-way  layout  situation.  The  new  HANOVA  procedures 
can  be  applied  to  many  other  business  simulation  experiments.  For 
example,  the  different  inventory  costing  methods  can  be  considered 
together  with  different  depreciation  methods  to  test  the  effects  of 
different  factors  on  profit  performance.  Other  applications  include 
different  audit  procedures  in  variable  sampling,  different  job  shop 
scheduling  schemes  and  different  advertising  strategies. 


FOOTNOTES 


Type  I  error  is  defined  as  the  error  of  rejecting  the  null  hypothesis 
when  it  is,  in  fact,  true.  Type  II  error  is  defined  as  the  error  of 
accepting  the  null  hypothesis  when  it  is  actually  false. 

Experiments  in  situations  where  variances  are  unknown  and  probably 
unequal  are  called  "heteroscedastic  experiments."  Kleijnen  and 
Naylor  [1969]  observed  that  "Only  in  rare  cases  can  the  assumption  of 
a  common  known  variance  be  expected  to  hold  with  computer  simulation 
experiments  with  models  of  business  and  economic  systems"  (pp.  609- 
610). 

The  reason  to  replicate  Lin's  multiple  objective  budgeting 


simulation  model  is  that  it  is  a  significant  improvement  over 

previous  accounting  simulations  in  terms  of  external  validity.  It  is 

a  more  sophisticated  model  with  an  improved  analytical  technique. 

2 

One  may  view  z  as  playing  the  role  of  O  /N,  standard  error  of  the 

mean,  in  the  traditional  r-way  layout  when  the  errors  have  equal 
2  2 

variance  0,0  is  known,  and  N  observations  are  taken  in  each  cell. 
Bishop  and  D  'dewicz  [1978]  developed  tables  for  choosing  z  values  in 
the  one-way  layout.  For  the  two-way  layout  z  values  are  related  to 
noncentrality  parameters  in  noncentral  chi-square  distributions. 


While  other  choices  of  the  [a_]  are  possible,  the  above  are  con¬ 
jectured  by  Dudewicz  and  Dalai  [1975]  to  be  robust  against 


nonnormality  of  the  errors. 

The  cut-off  points  (J®  ),  (F,a  ),  and  (F  J*  )  are  not  tabulated 

v  0,  n  1 ,  n  ’  2,  n 

’  o  ’  o  ’  o 


in  the  literature  yet,  and  thus  an  approximation  is  called  for.  It  has 
been  shown  by  Bishop  and  Dudewicz  [1981  ]  that  the  limiting  distribution 
of  F.  is  noncentral  chi-square  with  1-1  degrees  of  freedom  and  non- 


2  2 

centrality  parameter  AQ  -  J  I  a.  /  z,  denoted  by  Xj_j  (Aq). 

i=l 

Note  that  the  closeness  of  the  approximation  depends  in  no  way  upon 

2  - 

the  cr  .  That  is,  F^ ,  F^,  and  F^  are  independent  of  unknown 
population  variances. 

See  Lin  [1978]  for  the  description  of  the  overall  model  in  terms  of 
planning,  operations,  and  performance  evaluation  processes.  Lin 
[1980]  also  showed  examples  of  ex  post  accounting  variance  analysis 
under  both  goal  programming  and  multiple  objective  linear  program¬ 
ming  planning  models. 


2  2 

^  2  ^  2 

That  is,  Z  a.  and  Z  p  .  are  each  assumed  to  be  $30  for  profit, 
i=l  1  j=l  J 

and  each  $100  for  sales. 

The  initial  sample  size  should  be  large  enough  to  allow  Central  Limit 
Theorem-type  effects  to  take  hold,  and  large  enough  to  keep  the 
multiplier  z,  used  to  set  the  second  stage  sample  sizes,  relatively 
small.  A  number  of  around  20  seems  desirable,  while  a  number  less 
than  10  is  not  recommended. 

In  Lin's  [1978]  study,  the  sample  size  of  the  profit  variable  for  all 


four  modes  was  126.  The  power  of  that  test  was  around  0.80. 
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Table  1 


INITIAL  SAMPLE  MEANS  AND  VARIANCES 
(no  =  30) 


Variables 

Mode  1 

(GP-TRAD) 

Mode  2 

(GP-EXPO) 

Mode  3 

(MOLP-TRAD) 

Mode  4 

(MOLP-EXPO) 

Actual 

Per 

Profit 

Period: 

Mean 

Variance 

762 

12309 

i 

! 

792 

11794 

812 

10078 

812 

10096 

Actual 

Sales 

Mean 

6220 

6364 

6539 

6537 

1  i  I 

Per  Period:  Variance  :  38682  33591  ;  39816  j  37998 


-15- 


Table  2 


SIMULATION  STATISTICS 


Variables  and  Modes 

Final 

Sample 

Size 

Weighting 

Coefficient 

a .  . 
ij 

Weighting 

Coefficient 

b.  . 

Generalized 

Cell  Mean 

X.  . 

ij- 

Other 

Statistics 

Profit: 

Mode  1:  GP-TRAD 

178 

1 

.0057 

762 

X. . .=  788 

Mode  2:  GP-EXPO 

171 

1 

796 

FQ  =  3.99 

Mode  3:  MOLP-TRAD 

146 

.0067 

.0069 

794 

Fj  =  4.85 

Mode  4:  MOLP-EXPO 

146 

.0063 

.0070 

797 

F2  =  3.37 

Sales: 

Mode  1:  GP-TRAD 

51 

1 

j  .0177 

■ 

6204 

A/ 

X.  . .  =  6388 

Mode  2:  GP-EXPO 

44 

]  .0213 

6330 

Fq  =  75.40 

Mode  3:  MOLP-TRAD 

52 

!  .0181 

|B 

6506 

Fx  =  5.54 

Mode  4:  MOLP-EXPO 

50 

1 

|  .0182 

» 

WM 

6510 

i 

F2  =  4.84 

